Ultrasound imaging system with a neural network for deriving imaging data and tissue information

ABSTRACT

An ultrasound system according to some embodiments may include an ultrasound transducer configured to transmit ultrasound pulses toward tissue and generate echo signals responsive to the ultrasound pulses, a channel memory configured to store the echo signals, a beamformer configured to generated beamformed signals responsive to the echo signals, a neural network configured to receive one or more samples of the echo signals or the beamformed signals and produce a first type of ultrasound imaging data, and a processor configured to generate a second type of ultrasound imaging data, wherein the one or more processors may be further configured to generate an ultrasound image based on the first type of ultrasound imaging data and the second type of ultrasound imaging data and to cause a display communicatively coupled therewith to display the ultrasound image.

RELATED APPLICATIONS

This application is the U.S. National Phase application under 35 U.S.C.§ 371 of International Application No. PCT/EP2018/050086, filed on Jan.3, 2018, which claims the benefit of both Provisional Application Ser.No. 62/522,134, filed Jun. 20, 2017, and Provisional Application Ser.No. 62/442,691, filed Jan. 5, 2017. These applications are herebyincorporated by reference herein.

TECHNICAL FIELD

The present disclosure pertains to ultrasound systems and methods whichutilize a neural network for deriving imaging data, tissue informationand diagnostic information from raw ultrasound echoes.

BACKGROUND

Ultrasound is a widely used imaging modality in medical imaging as itcan provide real-time non-invasive imaging of organs and tissue fordiagnosis, pre-operative care and planning, and post-operative patientmonitoring. In a conventional ultrasound imaging system, a transducerprobe transmits ultrasound toward the tissue to be imaged and detectsechoes responsive to the ultrasound. Acquired echo signals (alsoreferred to as radio frequency or RF signals) are passed through aseries of signal processing components, including for example abeamformer which combines raw channel data (e.g., RF signals frommultiple transducer elements) or partially beam-formed signals ofpatches of transducer elements into fully beamformed signals, ademodulator which extracts quadrature signals from the beamformedsignals, and one or more filters, to produce image data (e.g., pixelinformation that may be used to produce a 2D or 3D ultrasound image). Inaddition to providing an ultrasound image of the anatomy, existingultrasound system may be configured to perform a variety of other typesof processing to extract additional information from the echo signals.For examples, many ultrasound systems are configured to process thebeamformed signals, typically in parallel with processing for anatomyimage formation, for extraction of Doppler data in order to provide flowinformation. To do so, conventional systems may utilize a particularlyconfigured signal processor to derive the Doppler frequency from aplurality of temporally spaced samples of the echo signals. Othersystems may be configured to perform elastography, vector flow imaging,and other types of imaging as may be desirable for a variety of clinicalapplications. For these purposes, conventional systems may includeparticularized hardware components (e.g., pre-programmed processors)which are designed to perform a specific sequence of signal and imageprocessing steps.

As such, conventional ultrasound system, while providing a significantadvancement in medical imaging, may still benefit from furtherimprovements. For example, conventional signal processing componentsrely on and implement model-based algorithms, some of which may beimperfect and thus only provide approximations. Limitations ofpre-programmed algorithms, as well as hardware limitations, mayintroduce image artifacts or other shortcomings in the output of aconventional system. The hard-wiring or pre-programming of specificmodels or signal processing paths into a system may render it lessflexible for future improvements and adaptation to the needs of a user.Therefore, improvements in this area may be desirable. Also, currentultrasound systems generally require the user to carefully watch theultrasound system display, coordinate transducer movements andmanipulate user controls to precisely record the desired anatomy orpathology of interest. After capturing the desired images the user willtypically review the images and manually annotate specific anatomy orpathology. Techniques for simplifying operation of an ultrasound imagingsystem without sacrificing image and/or diagnostic information qualitymay thus also be desirable.

SUMMARY

The present disclosure pertains to ultrasound systems and methods whichutilize a neural network (e.g., a machine-trained algorithm or hardwareimplemented network of artificial neurons or nodes) for deriving imagingdata and/or a variety of other tissue information, such as tissue typecharacterization information, qualitative or quantitative diagnosticinformation, and other types of clinically relevant information) fromraw ultrasound echo signals or from fully or partially beam-formed RFsignals. In some embodiments, the neural network may be a deep neuralnetwork capable of analyzing patterns using a with a multi-dimensional(2- or more dimensional) data set, which may also be thought of as alocalized data sets, and where the location of data within the data setand the data values may both contribute to the analyzed result.

An ultrasound system according to some embodiments may include anultrasound transducer configured to transmit ultrasound pulses towardtissue and generate echo signals responsive to the ultrasound pulses, achannel memory configured to store the echo signals, a beamformerconfigured to generated beamformed signals responsive to the echosignals, a neural network configured to receive one or more samples ofthe echo signals or the beamformed signals and produce a first type ofultrasound imaging data, and a processor configured to generate a secondtype of ultrasound imaging data, wherein the one or more processors maybe further configured to generate an ultrasound image based on the firsttype of ultrasound imaging data and the second type of ultrasoundimaging data and to cause a display communicatively coupled therewith todisplay the ultrasound image.

In some embodiments, the ultrasound imaging system may be configured toproduce B-mode imaging data as the second type of imaging data, and toproduce Doppler imaging data, vector flow imaging data, elastographyimaging data, tissue type characterization data, wall shear stress of ananatomical structure containing a fluid therein, tissue compositiondata, ultrasound contrast agent information, plaque characterizationdata, one or more diagnostic indicators associated with the B-modeimaging data, or any combinations thereof as the first type of imagingdata.

In some embodiments, the neural network may include a deep neuralnetwork (DNN) or a convolutional neural network (CNN). In someembodiments, the neural network may be implemented in hardware,software, or a combination thereof. For example, the neural network maybe implemented, at least in part, in a computer-readable mediumcomprising executable instructions, which when executed by a neuralnetwork processor coupled to the channel memory, the beamformer, orboth, cause the neural network processor to perform a machine-trainedalgorithm to produce the first type of ultrasound imaging dataresponsive to the one or more samples of the echo signals or thebeamformed signals.

In some embodiments, the neural network may include a data selectorconfigured to select samples of the stored echo signals or thebeamformed signals as input to the neural network. In some embodiments,the data selector may be configured to selectively couple either samplesof echo signals or samples of beamformed signals to the neural networkresponsive to a control signal received by the data selector. In someembodiments, the control signal may be generated responsive to userinput such as to enable the user to select the type of input providedand corresponding operational mode of the neural network. In someembodiments, the neural network may be configured to additionallyreceive auxiliary data as input and the auxiliary data may includeultrasound transducer configuration information, beamformerconfiguration information, information about the medium, or combinationsthereof. In such embodiments, the first type of imaging data may beestimated by the neural network further based on the auxiliary data.

In some embodiments, the neural network may be operatively associatedwith a training algorithm configured to receive an array of traininginputs and known outputs, wherein the training inputs comprise echosignals, beamformed signals, or combinations thereof associated with aregion of imaged tissue and the known outputs comprise known propertiesof the region of imaged tissue. In some embodiments, the trainingalgorithm may be configured to utilize, with training inputs comprisingultrasound data, known outputs obtained using an imaging modality otherthan ultrasound. In some embodiments, the neural network may beconfigured to process the input data in accordance with one of aplurality of operational modes, which may be selected responsive to userinput or automatically set by the ultrasound system based on an imagingmode of the ultrasound system during acquisition of the echo signals. Insome embodiments, the neural network may be configured to predict a fatcontent of the tissue based on the input data without use of the secondtype of imaging data. In some embodiments, the neural network may beconfigured to predict flow properties of a fluid contained in ananatomical structure of the tissue based on temporally successivesamples of the input data without the use the quadrature signalsproduced by the image processing circuit. In some embodiments, theneural network may be configured to produce predicted beamformed signalsbased on samples of the echo signals, and to use the predictedbeamformed signals to generate the first type of imaging data.

A method of ultrasound imaging in accordance with some embodiments mayinclude generating echo signals responsive to ultrasound transmitted bya transducer operatively coupled to an ultrasound system, storing theecho signals in channel memory, beamforming a plurality of the echosignals from the channel memory to produce beamformed signals, couplingsamples of the echo signals, the beamformed signals, or a combinationthereof, to a neural network trained to output a first type of imagingdata responsive to the samples of the echo signals or the beamformedsignals, coupling the beamformed signals to a processor configured togenerate a second type of imaging data responsive to the beamformedsignals, and generating an ultrasound image based on the first type ofimaging data and the second type of imaging data. One or more frames ofultrasound images generated responsive to both the first type and secondtype of imaging data may be provided to a display, to storage (e.g.,persistent storage or a cineloop memory), or another type of outputdevice for real-time use or subsequent use (e.g., in subsequent trainingof the neural network). In some embodiments, the second type of imagingdata may be B-mode imaging data and the first type of imaging data mayinclude flow imaging data, tissue stiffness imaging data, wall shearstress of an anatomical structure containing a fluid therein, tissuecomposition data, ultrasound contrast agent information, plaquecharacterization data, one or more diagnostic indicators associated withthe B-mode imaging data, or combinations thereof.

In some embodiments, coupling samples of the echo signals, thebeamformed signals, or a combination thereof to the neural network mayinclude coupling an input (e.g., an array) comprising samples of theecho signals or samples of the beamformed signals corresponding to alocation within a region of imaged tissue to the neural network andpropagating the input through the neural network to estimate pixel dataassociated with the location within the region of imaged tissue. In someembodiments, coupling samples of the echo signals, the beamformedsignals, or a combination thereof to the neural network may includecoupling to the neural network an input comprising samples of echosignals or samples of beamformed signals from temporally sequentialtransmit and receive cycles received from a location within a region ofimaged tissue and propagating the input through the neural network toestimate a velocity of flow at the location. In some embodiments, themethod may include selectively coupling, responsive to user input,either samples of the echo signals or samples of the beamformed signalsas input data to the neural network, and selecting a correspondingoperational mode of the neural network based on the input data. In someembodiments, the method may further include training the neural network.For example, the method may include providing at least some of thesecond type of imaging data to the neural network during a training modeof the neural network. In some embodiments, the method may furtherinclude training the neural network using imaging data obtained by animaging modality other than ultrasound.

Any of the methods described herein, or steps thereof, may be embodiedin non-transitory computer-readable medium comprising executableinstructions, which when executed may cause a processor of a medicalimaging system to perform method or steps embodied therein.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a block diagram of an ultrasound system in accordance withprinciples of the present inventions.

FIG. 2 shows a delay and sum beamforming technique.

FIG. 3 shows aspects of input data selection for a neural network inaccordance with principles of the present inventions.

FIG. 4 shows a block diagram of an input data selector in accordancewith principles of the present inventions.

FIG. 5 shows another block diagram of an input data selector inaccordance with further principles of the present inventions.

FIG. 6 is a flow diagram of a process of producing ultrasound images inaccordance with the principles of the present inventions.

DESCRIPTION

The following description of certain exemplary embodiments is merelyexemplary in nature and is in no way intended to limit the invention orits applications or uses. In the following detailed description ofembodiments of the present systems and methods, reference is made to theaccompanying drawings which form a part hereof, and in which are shownby way of illustration specific embodiments in which the describedsystems and methods may be practiced. These embodiments are described insufficient detail to enable those skilled in the art to practice thepresently disclosed systems and methods, and it is to be understood thatother embodiments may be utilized and that structural and logicalchanges may be made without departing from the spirit and scope of thepresent system. Moreover, for the purpose of clarity, detaileddescriptions of certain features will not be discussed when they wouldbe apparent to those with skill in the art so as not to obscure thedescription of the present system. The following detailed description istherefore not to be taken in a limiting sense, and the scope of thepresent system is defined only by the appended claims.

An ultrasound system according to the present disclosure may utilize aneural network, for example a deep neural network (DNN), a convolutionalneural network (CNN) or the like, to bypass certain processing steps inconventional ultrasound imaging. In some examples, the neural networkmay be trained using any of a variety of currently known or laterdeveloped machine learning techniques to obtain a neural network (e.g.,a machine-trained algorithm or hardware-based system of nodes) that isable to derive or calculate the characteristics of an image for displayfrom raw channel data (i.e., acquired radio frequency (RF) echo signals)or in some cases, from partially- or fully-beamformed signals. Neuralnetworks may provide an advantage over traditional forms of computerprogramming algorithms in that they can be generalized and trained torecognize data set features by analyzing data set samples rather than byreliance of specialized computer code. By presenting appropriate inputand output data to a neural network training algorithm, the neuralnetwork of an ultrasound system according to the present disclosure canbe trained to produce image data (e.g., flow imaging data) and deriveother types of tissue information (e.g., tissue content or type,strain/stress data, identification of specific anatomical structures,within the imaged region, etc.) without the need for aphysically-derived model to guide system operation.

FIG. 1 shows an example ultrasound system in accordance with principlesof the present invention. The system 100 may include or be operativelycoupled to an ultrasound transducer 113 configured to transmitultrasound pulses toward a medium and generate echo signals responsiveto the ultrasound pulses. The ultrasound system 100 may include channelmemory 121 configured to store the acquired echo signals (raw RFsignals), and a beamformer 122, which may be configured to performtransmit and/or receive beamforming and which may include a beamformermemory 123 configured to store beamformed signals generated responsiveto the acquired echo signals. In some embodiments, the system 100 mayinclude or be communicatively coupled to a display 138 for displayingultrasound images generated by the ultrasound system 100.

The ultrasound transducer 113 may include an ultrasound transducer array114, which may be provided in a probe 112, for example a hand-held probeor a probe configured to be at least partially controlled by a computer(e.g., a machine-actuated probe). In some examples, the array 114 may beimplemented using a plurality of patches, each comprising a sub-array oftransducer elements and the array 114 may be configured to beconformably placed against the subject to be imaged. The array 114 isoperable to transmit ultrasound toward a region of interest and toreceive echoes for imaging the region of interest (ROI). A variety oftransducer arrays may be used, e.g., linear arrays, curved arrays, orphased arrays. The array 114 may include, for example, a two dimensionalarray of transducer elements capable of scanning in both elevation andazimuth dimensions for 2D and/or 3D imaging.

The array 114 may be coupled to a microbeamformer 116, which may belocated in the probe 112 or in an ultrasound system base (e.g., in acart-based system such as the SPARQ or EPIQ ultrasound system providedby Philips. The microbeamformer 116 may control the transmission andreception of signals by the array 114. The array 114 may be coupled tothe ultrasound system base via the microbeamformer 116, which may becoupled (via a wired or wireless connection) to a transmit/receive (T/R)switch 118 typically located in the base. The T/R switch 118 may beconfigured to switch between transmission and reception, e.g., toprotect the main beamformer 122 from high energy transmit signals. Insome embodiments, the functionality of the T/R switch 118 and otherelements in the system may be incorporated within the probe, such as aprobe operable to couple to a portable system, such as the LUMIFY systemprovided by PHILIPS. The probe 112 may be communicatively coupled to thebase using a wired or wireless connection. The probe 412 may becommunicatively coupled to the base using a wired or wirelessconnection. In some embodiments, the transducer, the channel memory, andhardware storing the neural network can be located in the probe, and adisplay for displaying images created by the neural network can becommunicatively coupled to the probe. For example, the display can becoupled via a cable to the probe or via wireless communication, in whichthe probe can include a wireless transmitter to send the image data tothe display. In certain embodiments, the system can include a graphicsprocessing unit (GPU) to fully or partially train the neural network inthe system. For example, a GPU can be located in a probe with thetransducer, the channel memory and the hardware storing the neuralnetwork. Alternatively, the GPU can be located separately from theprobe, such as being located in a tablet or other computing device, suchas a smart phone.

The ultrasound transducer 113 may be configured to acquire echo signalsresponsive to ultrasound signals transmitted toward a medium to beimaged (e.g., tissue). As described, the transducer 113 may include anarray of elements capable, under control from the transmit/receivecontroller 120, to transmit pulses of ultrasound toward the medium anddetect echoes responsive to the transmit pulses. The transmit/receivecontroller 120 controls the transmission of ultrasound signals by thetransducer 113 and the reception of ultrasound echo signals byindividual elements or groups of elements of the array (e.g., in thecase of a transducer including a microbeamformer (μBF) 116). Thetransmit/receive controller 120, in controlling the transmission andreception of signals, may receive input from the user's operation of auser interface 124. The user interface 124 may include one or more inputdevices such as a control panel 142, which may include one or moremechanical controls (e.g., buttons, encoders, etc.), touch sensitivecontrols (e.g., a trackpad, a touchscreen, or the like), and other inputdevices. Another function which may be controlled by thetransmit/receive controller 120 is the direction in which beams aresteered. Beams may be steered straight ahead from (orthogonal to) thetransmission side of the array 114, or at different angles for a widerfield of view. Echo signals received by the ultrasound transducer 113may be coupled to channel memory 110, which receives and stores theacquired echo signals. The channel memory 110 may be configured to storeper-element or group (in the case of microbeamformed signals) echosignals (also referred to as raw RF signals or simply RF signals, orper-channel data). The pre-channel data may be accumulated in memoryover multiple transmit/receive cycles.

The system 100 may further include a beamformer 122, which is configuredto receive the acquired echo signals and produce beamformed RF signals.The beamformer 122 may combine individual echo signals or partiallybeamformed signals from groups of transducer (e.g., in the case of atransducer associated with a microbeamformer) into a fully beamformedsignal. For example, the beamformer 122 may perform delay and sumbeamforming as shown in FIG. 2. As shown in FIG. 2, the beamformer(e.g., beamformer 122) may receive per-channel RF signals 206, whichcorrespond to echoes 211 from a reflector 210 as detected by elements205 of the array. The raw RF signals 206 are delayed by an appropriateamount of time 207 to temporally align them (as shown at 209) and thencombined (as shown at 215) into a beamformed signal 217, which may alsobe referred to as beamformed RF signal or summed RF signal. In somecases, the temporally aligned signals may be multiplied by a factor (asshown at 208) before they are summed. In some cases, a microbeamformermay be included, for example in the transducer probe, which performspartial beamforming of signals received by patches of elements (e.g., asubset of the elements detecting echoes in any given transmit/receivecycle) and thereby reduces the number of channel inputs into the mainbeamformer. In such cases, the main beamformer (e.g., beamformer 122)may produce fully beamformed signals corresponding to a scan line withinthe field of view from the partially beamformed signals. In someembodiments, the beamformer (and/or microbeamformer, if included) may beconfigured to use, alternatively or additionally, other techniques,including but not limited to, dual apodization with cross-correlation,phase coherence imaging, capon beamforming and minimum variancebeamforming, all operating on the per-channel data to combine theinformation from the echo signals and form an image line of thebackscattered ultrasound energy from tissue.

Referring back to FIG. 1, the beamformed signals produced by thebeamformer may be coupled to further downstream signal processingcomponents (e.g., processor 150) for generating one or more types ofultrasound imaging data, for example imaging data for producing agrayscale image of the scanned anatomy (e.g., a B-mode image). Theprocessor 150 may be implemented in software and hardware componentsincluding one or more CPUs, GPUs, and/or ASICs specially configured toperform the functions described herein for generating ultrasound imagesand providing a user interface for display of the ultrasound images. Insome embodiments, samples of the beamformed signals may, alternativelyor additionally, be coupled to the neural network 160, which may betrained to produce one or more other types of imaging data such as flowimaging data, tissue elasticity imaging data, and others. The imagingdata produced by the processor 150 may be used to generate an ultrasoundimage which also includes imaging data output from the neural network160, e.g., as described further below. In some embodiments, thebeamformed signals may be stored in beamformer memory 123, over one ormultiple transmit/receive cycles, until they are used for imagegeneration (e.g., by the neural network and/or by the one or more imageprocessors) or tissue characterization.

As shown in the example in FIG. 1, the system 100 may include one ormore processing components, collectively referred to as processor 150,which is configured to receive beamformed signals and generate one ormore types of imaging data for producing an ultrasound image (e.g., aB-mode image). For example, the beamformed signals may be coupled to asignal processor 126, which is configured to process the beamformedsignals in various ways, such as by bandpass filtering, decimation, Iand Q component separation, and harmonic signal separation. The signalprocessor 126 may also perform additional signal enhancement such asspeckle reduction, signal compounding, and noise elimination. Theprocessed signals may be coupled to a B-mode processor 128 for producingB-mode imaging data. The B-mode processor 128 can employ amplitudedetection for the imaging of structures in the body. The signalsproduced by the B-mode processor 128 may be coupled to a scan converter130 and a multiplanar reformatter 132. The scan converter 130 may beconfigured to arrange the signals in the spatial relationship from whichthey were received in a desired image format. For instance, the scanconverter 130 may arrange the signals into a two dimensional (2D)sector-shaped format, or a pyramidal or otherwise shaped threedimensional (3D) format. The multiplanar reformatter 132 can convertechoes which are received from points in a common plane in a volumetricregion of the body into an ultrasonic image (e.g., a B-mode image) ofthat plane, for example as described in U.S. Pat. No. 6,443,896(Detmer). A volume renderer 134 may generate an image of the 3D datasetas viewed from a given reference point, e.g., as described in U.S. Pat.No. 6,530,885 (Entrekin et al.).

Additionally and optionally, signals from the signal processor 126 maybe coupled to a Doppler processor 144, which may be configured toestimate the Doppler shift and generate Doppler image data. The Dopplerimage data may include colorflow data which may be overlaid with B-mode(or grayscale) image data for displaying a conventional duplexB-mode/Doppler image. In some examples, the Doppler processor 144 mayinclude a Doppler estimator such as an auto-correlator, in whichvelocity (Doppler frequency) estimation is based on the argument of thelag-one autocorrelation function and Doppler power estimation is basedon the magnitude of the lag-zero autocorrelation function. Motion canalso be estimated by known phase-domain (for example, parametricfrequency estimators such as MUSIC, ESPRIT, etc.) or time-domain (forexample, cross-correlation) signal processing techniques. Otherestimators related to the temporal or spatial distributions of velocitysuch as estimators of acceleration or temporal and/or spatial velocityderivatives can be used instead of or in addition to velocityestimators. In some examples, the velocity and power estimates mayundergo threshold detection to reduce noise, as well as segmentation andpost-processing such as filling and smoothing. The velocity and powerestimates may then be mapped to a desired range of display colors inaccordance with a color map. The color data, also referred to as Dopplerimage data, may then be coupled the scan converter 130 where the Dopplerimage data is converted to the desired image format and overlaid on theB-mode image of the tissue structure containing the blood flow to form acolor Doppler image.

Additionally and optionally, the system 100 may include a vector flowprocessor 146, which may be configured to extract beam-angle-independentvelocity information from the signals generated responsive to thedetected echoes. For example, the vector flow processor 146 may beconfigured to estimate the beam-angle-independent velocity components offlow within an imaged bodily structure (e.g., a blood vessel) using thetransverse oscillation method or synthetic aperture method (e.g., asdescribed by Jensen et al., in “Recent advances in blood flow vectorvelocity imaging,” 2011 IEEE International Ultrasonics Symposium, pp.262-271, the disclosure of which is incorporated herein by reference inits entirety for any purpose), or any other currently known or laterdeveloped vector flow estimation and imaging technique.

Similar to the imaging data from the B-mode processor, the imaging dataproduced by any of these additional optional image data processors maybe coupled to the scan converter 130 for arranging the signals in thespatial relationship from which they were received in a desired imageformat (e.g., a two dimensional (2D) sector-shaped format, or apyramidal or otherwise shaped three dimensional (3D) format) and to themultiplanar reformatter 132 for extracting a set of the signals receivedfrom points in a common plane in a volumetric region of the body into anultrasonic image of that plane. In the case of 3D imaging, the signalsmay additionally or alternatively be coupled to the volume renderer 134may generate an image of the 3D dataset as viewed from a given referencepoint. Thus, it will be understood that the processor 150 may includeone or more signal processing paths, circuits, or specificallyprogrammed software components to produce B-mode imaging data, Dopplerimaging data, vector flow imaging (VFI) data, etc. as may be desired,which data may be arranged in a desired display format includingcombining the data for duplex (e.g., an overlay of Doppler and B-modedata or of B-mode and stress/strain data) or triplex displays (e.g., anoverlay of Doppler and B-mode in one image window concurrently displayedwith a second image window showing a graphical display such as an M-modeimage or a spectral Doppler image).

In accordance with principles of the present invention, the system 100may include a neural network 160. The neural network 160 may beconfigured to receive one or more samples of the echo signals, one ormore samples of the beamformed signals, or a combination thereof, asinput and produce another type of ultrasound imaging data responsive tothe input. The neural network 160 may be implemented in hardware (e.g.,using hard-wired circuitry for the artificial nodes of the network)and/or software components (e.g., using executable instructions whichprogram one or more processors to implement a machine-trainedalgorithm). The neural network 160 may be trained to propagate the input(e.g., samples of raw echo signals and/or samples of beamformed signals)through the network of nodes to obtain predicted or estimated imagingdata, which may subsequently be further processed for display. In somecases the network may be trained to operate in any one of a plurality ofoperational modes and may produce, responsive to the same input, adifferent type of imaging data or output other tissue informationdepending on the operational mode of the network. The mode may beselective (e.g., responsive to user input, or automatically selected bythe system).

As described, the neural network 160 may be hardware- (e.g., neurons arerepresented by physical components) or software-based (e.g., neurons andpathways implemented in a software application), and can use a varietyof topologies and learning algorithms for training the neural network toproduce the desired output. For example, a software-based neural networkmay be implemented using a processor (e.g., single or multi-core CPU, asingle GPU or GPU cluster, or multiple processors arranged forparallel-processing) configured to execute instructions, which may bestored in computer-readable medium, and which when executed cause theprocessor to perform a machine-trained algorithm for producingultrasound images and/or outputting tissue information from one or moreof the above identified inputs. The ultrasound system may include adisplay or graphics processor, which is operable to arrange theultrasound image and/or additional graphical information, which mayinclude annotations, tissue information, which may also be output by theneural network, and other graphical components, in display window fordisplay on the display of the ultrasound system. In some embodiments,the ultrasound images and tissue information may additionally beprovided to a storage device, such as a picture archiving andcommunication system (PACS) or another local or remote/networked storagedevice, for reporting purposes or future machine training (e.g., tocontinue to enhance the performance of the neural network). In yetfurther examples, imaging data obtained from a variety of differentimaging modalities (e.g., magnetic resonance imaging (MRI), computedtomography (CT), or another), which may be stored in PACS, mayalternatively or additionally be used to train the neural network. Aswill be appreciated, systems according to the present disclosure mayinclude a two-way communication link coupling the system, and morespecifically the neural network to source(s) of training data (e.g., astorage device) and/or to other machine-trained systems for ongoingfeedback and training.

In some embodiments, the neural network 160 may be configuredspecifically to produce imaging data and/or any desired tissueinformation other than B-mode imaging data. For example, the neuralnetwork may be trained to provide flow imaging data (e.g., beam-angledependent or beam-angle independent velocity information) directly fromthe echo signals and/or beamformed signals, while the system produces ananatomy image for overlay therewith using the pre-programmed ormodel-based processing components in processor 150. The B-mode imagingdata may then be combined (in this case, overlaid) with the flow imagingdata to produce an ultrasound image similar to a conventional Dopplerimage showing a color-coded flow map (or in the case of VFI, showing avector field) onto a grayscale anatomy image. In other embodiments, theneural network may be trained to provide elastography imaging data(e.g., tissue stiffness information) directly from the echo signalsand/or beamformed signals, while the system produces an anatomy imagefor overlay therewith using the pre-programmed or model-based processingcomponents in processor 150. In conventional elastography, the tissuestiffness information is color coded similar to how flow information maybe color coded for display. In the current example, the elastographyimaging data (e.g., tissue stiffness information) may be coupled to theprocessor 150 (e.g., to the scan converter and the multiplanarreformatter or volume renderer) to arrange the tissue stiffnessinformation in a desired format for display with the anatomy imageproduced based on B-mode data output by the B-mode processor. Othertypes of imaging data and/or tissue information may be estimateddirectly from the echo signals and/or beamformed signals by the neuralnetwork bypassing standard image formation techniques and reliance ondownstream signal processing. In this manner, the imaging data and/ortissue information output by the neural network may not be negativelyaffected by image artefacts and may thus be more accurate, especiallywith further training of the neural network over time. The use of aneural network for generating imaging data and tissue information in avariety of clinical applications, some of which are described furtherbelow, may be implemented in accordance with the examples herein.

To train a neural network 160 according to the present disclosure,training sets which include multiple instances of input arrays andoutput classifications, {Xi,Yn}, may be presented to the trainingalgorithm(s) of the neural network 160 (e.g., an AlexNet trainingalgorithm, as described by Krizhevsky, A., Sutskever, I. and Hinton, G.E. “ImageNet Classification with Deep Convolutional Neural Networks,”NIPS 2012 or its descendants). In the training data set, the input data[Xi] may include per-channel echo signals, e.g., as illustrated in FIG.3, optionally together with auxiliary data, described further below, andthe output data [Yi] may include any known properties of the tissuecorresponding to the sample of echo signals (e.g., known velocities inthe case of blood flow or other tissue motion imaging, knownstrain/stress values, or echo intensity data for producing anatomyimaging information, etc.). The input [Xi] and output [Yi] data of thetraining data sets may be acquired by an ultrasound imaging system whichhas components for conventional ultrasound imaging or an imaging systemconfigured for another type of imaging modality (e.g., an MRI scanner,CT scanner, and others). In some embodiments the system 100 may alsoinclude conventional beamforming, signal and image processing componentsto acquire input and output data sets for use in producing combinedimages or for providing additional training sets to the trainingalgorithm associated with neural network 160. For example, differenttypes of tissue may be scanned (e.g., ultrasonically scanned) using atransducer which is operatively associated with a spatial localizationsystem (e.g., an EM or ultrasonically tracked probe), which canspatially correlate the point or region of interest of tissue beingscanned to the output data (e.g., the imaging data and/or tissuecharacterization information to be used as the output in the trainingset). In further examples, the neural network 160 may be trained using asuitable ultrasound simulation such as the Field II program (asdescribed by J. A. Jensen: A Model for the Propagation and Scattering ofUltrasound in Tissue, J. Acoust. Soc. Am. 89, pp. 182-191, 1991), whichtakes as input the spatial distribution of points representingscatterers in an image field together with data about the geometry ofthe ultrasound transducer and the transmitted pulse and outputs computedultrasonic data representing the per-element echo signals (also referredto as simulated echoes). The system 100 may use this type of data fortraining purposes, e.g., by using the simulated echoes and auxiliarydata about the transducer and transmitted pulses for one or more givenpoints in space and present them to the neural network as input trainingdata [Xi] with the corresponding output data [Yi] being the scattererdensities from the simulation. Other algorithms or techniques mayadditionally or alternatively be used for training the neural network160. Also, as noted, in some cases, the output data (e.g., imaging dataand/or known properties of tissue) may be obtained using an imagingmodality different from ultrasound, for example MRI, CT or others or anycombinations thereof. The neural network 160 may thus be trained toproduce imaging data, and in some case images of higher quality (e.g.,higher resolution) than may otherwise be possible through conventionalultrasound image processing directly from the RF signals.

A neural network training algorithm associated may be presented withthousands or even millions training data sets in order to train theneural network 160 to directly estimate or output imaging data or avariety of tissue properties based on the raw measurement data (i.e.,raw echo or beamformed RF signals) without reliance on an explicit modelof the input/output relationship (e.g., pre-programmed physics-basedmodels typically implemented in conventional beamformers, signalprocessors or imaging data processors of existing ultrasound systems).That is, hundreds, thousands, or millions of training data sets may bepresented to a machine learning algorithm to develop a network ofartificial neurons or nodes arranged in accordance with any one of avariety of topographies or models. The neurons of the neural network aretypically connected in layers and signals travel from the first (input)layer to the last (output) layer. With advancements in modern neuralnetworks and training algorithms, a neural network comprising hundredsof thousands to millions of neurons or nodes and connectionstherebetween may be developed. The signals and state of the artificialneurons in a neural network 160 may typically be real numbers, typicallybetween 0 and 1, and a threshold function or limiting function may beassociated with each connection and/or node itself, such that the signalmust equal or exceed the threshold/limit before propagating.

The output of the training process may be a set of weights (alsoreferred to as connection or node weights) which may be used by theneural network 160 during operation (e.g., to adjust the threshold orlimiting functions controlling propagation through the layers of theneural net). Once trained, the neural network 160 may be configured tooperate on any input array, Xk, to produce one or more output valuesthat can be interpreted loosely as a probability or confidence estimatethat Xk is a member of the output set Yn (e.g. that the sample of echosignals correspond to a set of pixel image data). The output sets, Yn,can also represent numerical value ranges. In this manner, a set of RFsignals may be provided as input to the neural network 160, the set ofRF signals corresponding to a subset of a given spatial locations (e.g.,a region of interest in the imaged tissue) within the medium and theneural network may provide as output a set of corresponding pixel datafor producing a portion of the image at the given spatial location. Insome examples, by changing the weights of the neural network, thissystem can be dynamically reconfigured to produce images of a widevariety of different characteristics. In some embodiments, the neuralnetwork 160 may be a deep-learning or simply deep neural network (DNN)and/or an adaptive neural network. In some embodiments, a deep neuralnetwork (DNN), such as a deep convolutional neural networks (deep CNN)also referred to as fully convolutional network (FCN), may be used tolocalize objects within an image on a pixel by pixel basis. In examples,the input training arrays, Xi, may be formed from any desired type ofimage data (e.g., flow image data, elastography image data) surroundingpoint in a region of interest in an image. Each training array may beclassified into one or more output sets or values based on the setmembership of the output point or pixel in question. As such, theultrasound system 100 may be configured to at least partially bypasscertain conventional signal processing to output imaging data forproducing ultrasound images, in some cases in combination with echointensity information derived through conventional signal processing.Rather, the estimation or prediction of values that form the imagingdata output by the neural network would be accomplished implicitlywithin the neural network.

In examples of blood flow and/or contrast imaging, temporal aspects maybe accounted for in the training of the neural network. If theultrasonic echoes, ei, as shown in FIG. 3, are selected to include dataacquired over multiple ultrasonic transmit receive cycles, then atraining process as described herein may be used to distinguishdifferent tissue types on the basis of the spatial and temporalcharacteristics of the echoes from multiple transmit events. In, such amanner, the neural network may be trained to produce imaging dataassociated with moving tissue (e.g., blood flow) and/or contrast media.Further, in examples of intravascular ultrasound (IVUS) imaging, such aswhen the neural network operates on echoes acquired from an arraysupported on an imaging catheter, the neural network may be trained toidentify and localize flowing flood from the 2D echo image data bytraining the neural network with per-channel echo data from multipleframes as training input (xi) and the corresponding segmented (in somecases manually) image data (tissue/blood) as training output (yi). Echosignals and/or beamformed signals acquired over multipletransmit/receive cycles may be used for other blood flow or other movingtissue detection and imaging applications.

As described, the output of the neural network 160 may be coupled to theprocessor 150 for combining with imaging data produced by conventionalsignal processing techniques. For example, the output of the neuralnetwork 160 may be coupled to the scan converter 130 and multiplanarreformatter 132 and/or volume renderer 134 for arranging the subsets ofpixel data received from the neural network, based on their spatialattributes, and presenting the imaging data in a desired format (e.g., a2D or 3D ultrasound image). In some examples, the imaging data (e.g.,pixel data) or tissue information provided by the neural network may bebuffered until sufficient amount of pixel data, for example, enough toconstruct a full frame of a color flow, vector flow, or an elastographyimage, has been output by the neural network. In some examples, prior topassing the output data to the processor 150, the output of the neuralnetwork may be passed through a data conditioner 164, which may beconfigured to spatially and temporally process the output data tohighlight certain spatial and/or temporal characteristics thereof. Insome examples, the data conditioner 164 may be configured to performmulti-resolution image processing.

Once combined the imaging data produced by the neural network has beencombined with that produced by processor 150, the data may beadditionally processed, in conventional manner to enhance the finalimage and/or add annotation, as previously described. The ultrasoundimage(s) may be displayed in real-time, e.g., on a display unit of theuser interface 124, buffered into a cineloop memory for displayingtemporal sequences of images, and/or exported to a storage device or aprinting system. Stored ultrasound images (orpre-formatted/pre-annotated imaging data) may be retrieved forsubsequent analysis and diagnosis, inclusion in a report and/or for useas training data. The ultrasound imaging data may be further processedusing conventional techniques to extract additional quantitative and/orqualitative information about the anatomy or characteristics of thetissue being scanned.

In some embodiments, the RF signals (e.g., from channel memory 110 orthe beamformer memory) may be coupled to the neural network 160 via aninput data selector 162. The input data selector 162 may be configuredto select, for each point or region of interest (ROI) in an imagingfield of view, a corresponding array of m RF signal samples (e.g., echosignal samples) from each or a subset of elements of the transducerarray. In some examples, the input data selector 162 may be configuredto select the samples such that the centers of the samples of RF signalscorrespond approximately to the round trip time delay and thus to thedepth of interest (see e.g., FIG. 3). As shown in the example in FIG. 3,m samples of echo signals e_(i) (i.e. per-channel data represented bye₁, e₂, e₃, . . . e_(n)) are shown to have been selected based on havingtheir centers corresponding to the depth of interest. In some examples,as long as the data segment lengths are long enough to include theinformation from each echo surrounding the depth of interest, it may notbe strictly necessary to center the depth of interest within the echosegments. In some embodiments, the data selector 162 may thus beconfigured to select a subset of echo signals from the acquired echosignals, which are associated with adjacent points within a region ofimaged tissue. After selection of the appropriate input data set,imaging and other tissue data extraction would be performed implicitlyby the neural network 160 without the reliance on conventionalbeamforming.

In some examples, the neural network 160 may be trained to operate in aplurality of modes based, at least in part, on the input data type(e.g., per-channel data, beamformed signals, quadrature data, imagingdata, or a combination thereof). For example, when using beamformed RFsignals, select samples of the beamformed RF signals (over single ormultiple transmit/receive cycles) may be coupled to the neural network160, e.g., via an input data selector 162, which is configured to selectthe appropriate sample of beamformed RF signals corresponding to theROI. In such examples, instead of or in addition to per-channel signals,the sample selector would select samples of RF signals corresponding toscanlines from the region of interest and as well as neighboring linesof sight, or they could may represent beamformed RF signalscorresponding to scanlines generated from successive transmit events.Ultrasound imaging data for the scanlines corresponding to the inputarray of beamformed RF signals can thus be implicitly estimated by theneural network directly from the beamformed RF signals.

The input data selector 162 may be further configured to activate theappropriate operational mode of the neural network 160, which may beresponsive to user input or which may be a pre-programmed default basedon the imaging mode during acquisition of the echo signals (e.g., flowimaging, elastography, etc.). FIGS. 4 and 5 show examples of input dataselectors 462 and 562 which may be operatively associated with neuralnetwork 160 to select the type and sample of input data and/or activatethe appropriate mode of the neural network. The input data selector 462or 562 may be used to implement the data selector 162 of the examplesystem in FIG. 1.

For example, referring to FIG. 4, the input data selector 462 mayinclude an input type selector which may selectively couple the type ofdata (e.g., echo signals or beamformed signals) to the neural networkresponsive to a control signal. The control signal may be generatedbased on user input. The input data selector 462 may include a sampleselector which selects and couples the appropriate sample of signals ofthe selected type as previously described. To that end, and as shown inthe example in FIG. 5, the input type selector 564 may include a modeselector 565 and a switch 567. The mode selector 565 may receive thecontrol signal and send a select (Sel) signal to the switch toselectively couple the samples of the appropriate type of input data tothe neural network 160. Additionally the input data selector 562 may beconfigured to activate the appropriate mode of the neural network, forexample by transmitting a mode control signal (e.g., from the modeselector 565) to the neural network 160.

In some embodiments, the neural network 160 may be trained to operate inone or a plurality of different modes further based on the type ofimaging data or tissue information that may be desired to be obtained.As described, the neural network 160 may be configured to outputdifferent types of imaging data responsive to the same input. For eachof these different types of imaging data or tissue information, thenetwork may be trained and thus include different propagation paths(e.g., layers of nodes and connection developed through appropriatetraining) and the propagation path or mode may be selected by the useror automatically invoked by the system depending on the imaging mode orapplication (e.g., blood flow imaging, fetal ultrasound imaging, etc.)

In some embodiments, the neural network of an ultrasound systemaccording to the present disclosure may be configured to performultrasonic tissue characterization, for example to characterize fatcontent, plaque, or for ultrasonic contrast imagining, e.g., bypresenting the neural network during a training phase with appropriatetraining data sets of inputs and known outputs, for example obtainedthrough conventional ultrasound imaging or through imaging using adifferent modality.

For example, in ultrasonic liver imaging, ultrasonic attenuation andback-scattering (i.e., tissue echogenicity) increases in proportion tofat content while speed of ultrasound correspondingly reduces. Byquantifying the ultrasound attenuation, echogenicity and/or speed fromthe beamformed RF echoes and correlating this attenuation with fatcontent, estimates of the fat content of the liver (or other tissue ororgans, in other applications) may be performed with ultrasound. Thecustomer-facing output of such a system may be quantitative (e.g., asingle value representing the fat fraction within the imaged tissue),which may be displayed onto an image of the anatomy (e.g., for aspecific point or region of interest) or it may be graphicallyrepresented, with each quantitative value being color-coded and overlaidon a 2D image or a 3D volume rendering of the liver (or other organ ortissue) similar to conventional overlays of blood flow or elastographyinformation. As described, a neural network

To train a neural network to extract tissue information pertaining totissue content (e.g., fat content), the neural network may be presentedwith training data sets including localized raw RF signals and/orbeamformed RF signals as inputs and the corresponding quantified tissueparameter (e.g., fat content or other type of tissue content), which maybe obtained via the ultrasound quantification method above or throughother imaging or non-imaging process capable of determining the tissuecontent of the tissue being imaged, as the known output. Onceappropriately trained, the neural network may be opearable to implicitlyextract this information directly from the raw RF signals and/orbeamformed RF signals without reliance on the ultrasound quantificationmethod used to initially obtain the training data.

In another example, plaque characterization may be enhanced by a neuralnetwork appropriately trained to replace existing vessel tissueclassification models that are preprogrammed in conventional ultrasoundsystems, such as may be used by intravascular ultrasound (IVUS)catheters to provide colorized tissue map of plaque composition withlumen and vessel measurements. For example, the VH algorithm provided byPhilips Volcano can be said to generally utilize beamformed ultrasoundRF signals from an IVUS catheter and analyze the short-time windowed RFspectral properties of these echo signals to classify the tissue intoone of several different categories such as fibrous tissue, necroticcore, dense calcium and fibro-fatty tissue. An image may then beprovided showing the distribution of these tissue types within a vesselwall. Thus, to train the neural network of a system according to thepresent disclosure to provide relevant vessel tissue classificationinformation, training data sets including IVUS-obtained RF signals maybe provided as input with corresponding known tissue classifications(e.g., fibrous, necrotic core, dense calcium, etc.) as known outputsduring a training phase of the neural network. Generally, raw RF signalsand/or beamformed RF signals and corresponding vascular pathology dataobtained using an existing IVUS system may be used to train anultrasonic imaging system with a neural network to estimate vasculartissue composition directly from the detected echoes and/or beamformedsignals, without the need for Fourier transforms and heuristictechniques that may currently be employed by conventional IVUS systems.

In yet further examples, the neural network may be trained tocharacterize tissue with respect to the presence of ultrasonic contrastagents. In ultrasonic contrast imaging, per-channel data frommulti-pulse sequences (e.g. power modulation) are typically beamformedand then combined to form an image representing the volume density ofmicrobubble contrast agents across the imaging field of view. The samemay be implicitly (without beamforming and/or explicitly calculating thevolume density) be achieved by a neural network which is trained withinput training data in the form of per-channel data and/or at leastpartially beamformed data and corresponding known volume density ofmicrobubble contrast agents.

In further examples, human hearts from multiple test subjects could bescanned using 1D or 2D array transducers and the resulting images and/or3D volumes could be segmented (manually or automatically) into regionsthat are either a) within a cardiac chamber; or b) comprising myocardialtissue. These images may be used to train the neural network 160 toperform cardiac chamber recognition. In examples, the input data, [xi],may be per-channel data and optionally auxiliary data as describedherein, while the output data would be a classification (i.e., either a)or b). In examples, the neural network may include an appropriatelytrained semantic classifier to perform this type of classification. Theso trained neural network may then be used to segment and identifycardiac chambers directly from the raw or beamformed data without havingto first reconstruct an image of the anatomy and without reliance onimage processing techniques. This segmentation information could be usedto suppress imaging artifacts, or it could be fed directly intoalgorithms to quantify ejection fraction or other clinical parameters.The system may be similarly trained to identify other types of tissue oranatomical structures (e.g., walls of vessels, lung/pleura interface)and quantify relevant clinical parameters associated therewith (e.g.,obtain a nuchal translucency measurement).

As shown in the examples in FIGS. 1 and 4, the neural network may beconfigured to receive auxiliary data, such as information about theprogramming of the beamformer 122, properties of the transducer 113(e.g., number, arrangement, and/or spacing of elements of the array,type of array, etc.), known information about the anatomy being imaged,and/or the spatial location of the point or region of interest (e.g., asmay be obtained by a transducer tracking system). Other types ofinformation, for example in the case of training sets from differentimaging modalities, may also be provided as auxiliary information to thetraining algorithm. In some embodiments, the neural network may receiveauxiliary data may be used during the training process to supplement thetraining data sets. Optionally, auxiliary data may also be provided tothe neural network during an imaging session.

FIG. 6 shows a flow diagram of a process in accordance with someexamples of the present disclosure. The process 600 may begin by storingthe acquired echo RF signals in channel memory, as shown in block 602.The RF signals stored in channel memory correspond to the echoesdetected from the tissue being image response to ultrasound transmittedby a transducer (e.g., transducer 113) operatively coupled to anultrasound system (e.g., system 100). In some embodiments, the methodmay include generating beamformed RF signals based on the echo signals,as shown in block 604. The beamformed signals may be generated byconventional beamforming techniques, for example using beamformer 122 ofsystem 100. In some embodiments, beamformed RF signals from one ormultiple temporally sequential transmit/receive cycles, may be stored inbeamformer memory. The beamformed signals may be coupled to one or moresignal and image processors (e.g., processor 150). For example, thebeamformed signals may be coupled, as shown in block 606, to a signalprocessor for extracting quadrature data (i.e., I/Q components of thesignal) which can be coupled, as shown in block 608 to a B-modeprocessor for generating an anatomy image (see block 610).

Samples of the echo RF signals and/or beamformed RF signals may becoupled to a neural network (e.g., neural net 160), as shown in block612, and may then propagate through the layers of the network to produceimaging data or any type of tissue information depending upon thetraining and/or operation mode of the network. That is, in someembodiments, one type of imaging data may be obtained throughconventional signal processing (e.g., signal processing path 601) andanother type of imaging data may be obtained through implicit orpredictive analysis of the input data (e.g., along neural network path613) directly from the raw RF signals or beamformed RF signals, as shownin block 612. In some examples, the neural network may be deep neuralnetwork (DNN) or a convolutional neural network (CNN), which may beimplemented in hardware (e.g., nodes corresponding to hardwarecomponents) or software (e.g., where nodes are represented usingcomputer code). In some embodiments, the coupling of samples of raw orbeamformed RF signals may be selective, e.g., responsive to user inputor automatically controlled by the system based on the imaging mode orclinical application. The neural network may be trained to operate in aplurality of different modes each associated with a type of input data(e.g., raw channel data or beamformed data), and thus a correspondingoperational mode of the neural network may be selected (automatically orresponsive to user inputs) based on the type of input data to the neuralnetwork. The imaging data and/or tissue information output by the neuralnetwork may include B-mode imaging data, Doppler imaging data, vectorflow imaging data, strain imaging data, wall As shown in block 616,ultrasound images, which include both the first type of imaging data andthe second type of imaging data, may be produce and displayed on adisplay unit operatively associated with the ultrasound system.

In some embodiments, the neural network may also receive auxiliary datafor use in producing the imaging data. During an operational mode, theneural network, which may be associated with a training algorithm, mayreceive training data sets, for example image data or other knowninformation produced by the system itself (e.g., obtained by processingecho signals along path 601) or from other imaging systems which may ormay not utilize ultrasound as the imaging modality.

In various embodiments where components, systems and/or methods areimplemented using a programmable device, such as a computer-based systemor programmable logic, it should be appreciated that the above-describedsystems and methods can be implemented using any of various known orlater developed programming languages, such as “C”, “C++”, “FORTRAN”,“Pascal”, “VHDL” and the like. Accordingly, various storage media, suchas magnetic computer disks, optical disks, electronic memories and thelike, can be prepared that can contain information that can direct adevice, such as a computer, to implement the above-described systemsand/or methods. Once an appropriate device has access to the informationand programs contained on the storage media, the storage media canprovide the information and programs to the device, thus enabling thedevice to perform functions of the systems and/or methods describedherein. For example, if a computer disk containing appropriatematerials, such as a source file, an object file, an executable file orthe like, were provided to a computer, the computer could receive theinformation, appropriately configure itself and perform the functions ofthe various systems and methods outlined in the diagrams and flowchartsabove to implement the various functions. That is, the computer couldreceive various portions of information from the disk relating todifferent elements of the above-described systems and/or methods,implement the individual systems and/or methods and coordinate thefunctions of the individual systems and/or methods described above.

In view of this disclosure it is noted that the various methods anddevices described herein can be implemented in hardware, software andfirmware. Further, the various methods and parameters are included byway of example only and not in any limiting sense. In view of thisdisclosure, those of ordinary skill in the art can implement the presentteachings in determining their own techniques and needed equipment toaffect these techniques, while remaining within the scope of theinvention. The functionality of one or more of the processors describedherein may be incorporated into a fewer number or a single processingunit (e.g., a CPU) and may be implemented using application specificintegrated circuits (ASICs) or general purpose processing circuits whichare programmed responsive to executable instruction to perform thefunctions described herein.

Although the present system may have been described with particularreference to an ultrasound imaging system, it is also envisioned thatthe present system can be extended to other medical imaging systemswhere one or more images are obtained in a systematic manner.Accordingly, the present system may be used to obtain and/or recordimage information related to, but not limited to renal, testicular,breast, ovarian, uterine, thyroid, hepatic, lung, musculoskeletal,splenic, cardiac, arterial and vascular systems, as well as otherimaging applications related to ultrasound-guided interventions.Further, the present system may also include one or more programs whichmay be used with conventional imaging systems so that they may providefeatures and advantages of the present system. Certain additionaladvantages and features of this disclosure may be apparent to thoseskilled in the art upon studying the disclosure, or may be experiencedby persons employing the novel system and method of the presentdisclosure. Another advantage of the present systems and method may bethat conventional medical image systems can be easily upgraded toincorporate the features and advantages of the present systems, devices,and methods.

Of course, it is to be appreciated that any one of the examples,embodiments or processes described herein may be combined with one ormore other examples, embodiments and/or processes or be separated and/orperformed amongst separate devices or device portions in accordance withthe present systems, devices and methods.

Finally, the above-discussion is intended to be merely illustrative ofthe present system and should not be construed as limiting the appendedclaims to any particular embodiment or group of embodiments. Thus, whilethe present system has been described in particular detail withreference to exemplary embodiments, it should also be appreciated thatnumerous modifications and alternative embodiments may be devised bythose having ordinary skill in the art without departing from thebroader and intended spirit and scope of the present system as set forthin the claims that follow. Accordingly, the specification and drawingsare to be regarded in an illustrative manner and are not intended tolimit the scope of the appended claims.

The invention claimed is:
 1. An ultrasound system comprising: anultrasound transducer configured to transmit ultrasound pulses towardtissue and generate echo signals responsive to the ultrasound pulses; achannel memory configured to store the echo signals; a beamformerconfigured to generate beamformed radiofrequency (RF) signals responsiveto the echo signals; a neural network configured to receive input datacomprising one or more samples of the echo signals or the beamformed RFsignals and produce a first type of ultrasound imaging data; a processorconfigured to generate a second type of ultrasound imaging data based onthe beamformed RF signals, wherein the processor is further configuredto generate an ultrasound image based on the first type of ultrasoundimaging data and the second type of ultrasound imaging data; and a dataselector configured to select a subset of the stored echo signals or thebeamformed RF signals as the one or more samples for input to the neuralnetwork.
 2. The ultrasound imaging system of claim 1, wherein the secondtype of ultrasound imaging data comprises B-mode imaging data, andwherein the first type of ultrasound imaging data comprises one ofDoppler imaging data, vector flow imaging data, elastography imagingdata, tissue type characterization data, wall shear stress of ananatomical structure containing a fluid therein, tissue compositiondata, ultrasound contrast agent information, plaque characterizationdata, one or more diagnostic indicators associated with the B-modeimaging data, or combinations thereof.
 3. The ultrasound system of claim1, wherein the neural network includes a deep neural network (DNN) or aconvolutional neural network (CNN).
 4. The ultrasound imaging system ofclaim 1, wherein the neural network is implemented, at least in part, ina computer-readable medium comprising executable instructions, whichwhen executed by a neural network processor coupled to the channelmemory, the beamformer, or both, cause the neural network processor toperform a machine-trained algorithm to produce the first type ofultrasound imaging data responsive to the one or more samples of theecho signals or the beamformed RF signals.
 5. The ultrasound imagingsystem of claim 1, wherein the data selector is configured toselectively couple one of the sample of echo signals or the sample ofthe beamformed RF signals to the neural network responsive to a controlsignal received by the data selector.
 6. The ultrasound imaging systemof claim 1, further comprising a display, and wherein the processor isfurther configured to cause the display to display the ultrasound image.7. The ultrasound imaging system of claim 1, wherein the neural networkis further configured to receive auxiliary data as input, the auxiliarydata including ultrasound transducer configuration information,beamformer configuration information, information about the medium, orcombinations thereof, and wherein the first type of ultrasound imagingdata provided by the neural network is further based on the auxiliarydata.
 8. The ultrasound imaging system of claim 1, wherein the neuralnetwork is operatively associated with a training algorithm configuredto receive an array of training inputs and known outputs, wherein thetraining inputs comprise echo signals, beamformed RF signals, orcombinations thereof associated with a region of imaged tissue and theknown outputs comprise known properties of the imaged tissue.
 9. Theultrasound imaging system of claim 8, wherein the known properties areobtained using an imaging modality other than ultrasound.
 10. Theultrasound system of claim 1, wherein the neural network is configuredto process the input data in accordance with one of a plurality ofoperational modes, which is selected responsive to user input orautomatically set by the ultrasound system based on an imaging mode ofthe ultrasound system during acquisition of the echo signals.
 11. Theultrasound system of claim 1, wherein the neural network is configuredto predict a fat content of the tissue based on the input data withoutuse of the second type of imaging data.
 12. The ultrasound system ofclaim 1, wherein neural network is configured to predict flow propertiesof a fluid contained in an anatomical structure of the tissue based ontemporally successive samples of the input data without the use thequadrature signals produced by the image processing circuit.
 13. Theultrasound system of claim 1, wherein the neural network is configuredto produce predicted beamformed signals based on samples of the echosignals, and to use the predicted beamformed signals to generate thefirst type of imaging data.
 14. An ultrasound system comprising: anultrasound transducer configured to transmit ultrasound pulses towardtissue and generate echo signals responsive to the ultrasound pulses; achannel memory configured to store the echo signals; a beamformerconfigured to generate beamformed radiofrequency (RF) signals responsiveto the echo signals; a neural network configured to receive input datacomprising one or more samples of the echo signals or the beamformed RFsignals and produce a first type of ultrasound imaging data, wherein theneural network is further configured to receive auxiliary data as input,the auxiliary data including ultrasound transducer configurationinformation, beamformer configuration information, information about themedium, or combinations thereof, and wherein the first type ofultrasound imaging data provided by the neural network is further basedon the auxiliary data; and a processor configured to generate a secondtype of ultrasound imaging data based on the beamformed RF signals,wherein the processor is further configured to generate an ultrasoundimage based on the first type of ultrasound imaging data and the secondtype of ultrasound imaging data.
 15. The ultrasound imaging system ofclaim 14, wherein the second type of ultrasound imaging data comprisesB-mode imaging data, and wherein the first type of ultrasound imagingdata comprises one of Doppler imaging data, vector flow imaging data,elastography imaging data, tissue type characterization data, wall shearstress of an anatomical structure containing a fluid therein, tissuecomposition data, ultrasound contrast agent information, plaquecharacterization data, one or more diagnostic indicators associated withthe B-mode imaging data, or combinations thereof.
 16. The ultrasoundimaging system of claim 14, further comprising a data selectorconfigured to select a subset of the stored echo signals or thebeamformed RF signals as the sample for input to the neural network. 17.The ultrasound system of claim 14, wherein the neural network isconfigured to process the input data in accordance with one of aplurality of operational modes, which is selected responsive to userinput or automatically set by the ultrasound system based on an imagingmode of the ultrasound system during acquisition of the echo signals.18. An ultrasound system comprising: an ultrasound transducer configuredto transmit ultrasound pulses toward tissue and generate echo signalsresponsive to the ultrasound pulses; a channel memory configured tostore the echo signals; a beamformer configured to generate beamformedradiofrequency (RF) signals responsive to the echo signals; a neuralnetwork configured to receive input data comprising one or more samplesof the echo signals or the beamformed RF signals and produce a firsttype of ultrasound imaging data, wherein the neural network isconfigured to process the input data in accordance with one of aplurality of operational modes, which is selected responsive to userinput or automatically set by the ultrasound system based on an imagingmode of the ultrasound system during acquisition of the echo signals;and a processor configured to generate a second type of ultrasoundimaging data based on the beamformed RF signals, wherein the processoris further configured to generate an ultrasound image based on the firsttype of ultrasound imaging data and the second type of ultrasoundimaging data.
 19. The ultrasound imaging system of claim 18, wherein thesecond type of ultrasound imaging data comprises B-mode imaging data,and wherein the first type of ultrasound imaging data comprises one ofDoppler imaging data, vector flow imaging data, elastography imagingdata, tissue type characterization data, wall shear stress of ananatomical structure containing a fluid therein, tissue compositiondata, ultrasound contrast agent information, plaque characterizationdata, one or more diagnostic indicators associated with the B-modeimaging data, or combinations thereof.
 20. The ultrasound imaging systemof claim 18, further comprising a data selector configured to select asubset of the stored echo signals or the beamformed RF signals as thesample for input to the neural network.